Baby Mental Life: Study 2

Preregistered analyses

“Baby Mental Life: Study 2” was conducted on MTurk on 2018-08-04.

Our planned sample was 300 participants, and we anticipated that roughly 80% of recruited participants would pass all of our attention checks, so we initially recruited 378 participants (on the idea that ~80% of 378 ~ 300 participants; note that for administrative purposes we need to recuit participants in batches that were divisible by 9). After filtering out participants who failed at least one of our attention checks, we ended up retaining fewer than 300 participants, so we recruited an additional 16 participants for a total of 394 people recruited. At each stage, we recruited women and men through separate studies, in hopes of acquiring a roughly equal split between genders.

In the end, we ended up with a sample of 304 participants who passed our attention checks, 237 of whom came from unique GPS coordinates.

For this first pass, these data INCLUDE participants where there is another participant with an identical set of GPS coordinates as recorded by Qualtrics. Excluding these participants would exclude 67 participants.

Each participant assessed children’s mental capacities at 13 target ages between the ages of 0 and 5 years. For each target, they rated 20 mental capacities on a scale from 0 (not at all capable) to 100 (completely capable).

For more details about the study, see our preregistration here.

NAs introduced by coercionattributes are not identical across measure variables;
they will be droppedJoining, by = "question_qualtrics"

Study 1 EFA

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

EFA

To test H1, we planned to conduct an exploratory factor analysis (EFA) collapsing across all 13 target characters (and treating an individual participant’s responses to each character as if they were independent data points) - see the preregistration for more details.

As with Study 1, we planned to examine three factor retention protocols in order to determine how many factors to retain: Parallel analysis, minimizing BIC, and a set of preset criteria outlined in Weisman et al. (2017). Here we look at each solution in turn.

We predicted that we’d see a similar factor structure to that found in Study 1.

Rotation choices

We planned to examine oblimin-rotated solutions (which allow factors to correlate), but you could examine other rotation options by selecting a different rotation type here.

Parallel analysis

How many factors to retain?

Parallel analysis suggests that the number of factors =  4  and the number of components =  4 
Call: fa.parallel(x = d_all, plot = F)
Parallel analysis suggests that the number of factors =  4  and the number of components =  4 

 Eigen Values of 

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

These factors look extremely similar to what we saw in Study 1 (see above). I (Kara) would say that H1 is strongly supported.

Which capacities are attributed to which targets?

We could look at factor scores using the Study 2 EFA to see which capacities were attributed to which targets. This is not the primary way we planned to investigate this - this was listed as a “follow-up analysis” - but I’m putting it here so that it’s in close proximity to the EFA results for ease of interpretation.

Age as numeric (raw)

Ignoring unknown aesthetics: y

Age as numeric (square-root-transformed)

Ignoring unknown aesthetics: y

Age as ordinal

Ignoring unknown aesthetics: y

And here’s a close look at all of the raw data (color-coded according to the Study 2 EFA results):

Joining, by = "capacity"

|==================================                               | 53% ~2 s remaining     
|===================================                              | 54% ~2 s remaining     
|====================================                             | 55% ~2 s remaining     
|=====================================                            | 58% ~2 s remaining     
|======================================                           | 60% ~2 s remaining     
|=======================================                          | 61% ~2 s remaining     
|========================================                         | 62% ~1 s remaining     
|=========================================                        | 63% ~1 s remaining     
|==========================================                       | 65% ~1 s remaining     
|===========================================                      | 67% ~1 s remaining     
|============================================                     | 68% ~1 s remaining     
|============================================                     | 69% ~1 s remaining     
|==============================================                   | 71% ~1 s remaining     
|===============================================                  | 73% ~1 s remaining     
|================================================                 | 74% ~1 s remaining     
|=================================================                | 76% ~1 s remaining     
|==================================================               | 77% ~1 s remaining     
|===================================================              | 78% ~1 s remaining     
|====================================================             | 80% ~1 s remaining     
|=====================================================            | 82% ~1 s remaining     
|======================================================           | 84% ~1 s remaining     
|=======================================================          | 85% ~1 s remaining     
|========================================================         | 87% ~0 s remaining     
|=========================================================        | 88% ~0 s remaining     
|==========================================================       | 90% ~0 s remaining     
|===========================================================      | 92% ~0 s remaining     
|===========================================================      | 92% ~0 s remaining     
|============================================================     | 93% ~0 s remaining     
|=============================================================    | 95% ~0 s remaining     
|===============================================================  | 97% ~0 s remaining     
|================================================================ | 99% ~0 s remaining     

Joining, by = c("capacity", "factor", "order")

Joining, by = "capacity"

Minimizing BIC

How many factors to retain?

Very Simple Structure
Call: vss(x = x, n = n, rotate = rotate, diagonal = diagonal, fm = fm, 
    n.obs = n.obs, plot = plot, title = title, use = use, cor = cor)
VSS complexity 1 achieves a maximimum of 0.85  with  1  factors
VSS complexity 2 achieves a maximimum of 0.96  with  2  factors

The Velicer MAP achieves a minimum of 0.02  with  4  factors 
BIC achieves a minimum of  -153.95  with  8  factors
Sample Size adjusted BIC achieves a minimum of  30.35  with  8  factors

Statistics by number of factors 

What are these factors?

convergence not obtained in GPFoblq. 1000 iterations used.

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

A more complex picture, but the first 4 factors look similar to what we get through parallel analysis. (I think something similar happened with Study 1, but we should go back and compare.)

Which capacities are attributed to which targets?

Ignoring unknown aesthetics: y

We’ll skip regression analyses and other plots for now.

Preset retention criteria

How many factors to retain?

[1] "Preset criteria suggest retaining 4 factors"

This gives the same solution as parallel analysis - 4 factors :)

Projecting into Study 1 factor space

I (Kara) made a big mistake in thinking through this: I thought we could project a 20-variable dataset into a 60-variable dataset using the predict.psych() function, but we can’t!

I’ve tried to hack together a way to do this, by replacing all missing values at random (either within the full range of the scale, or around the midpoint, or near 0) - but I haven’t gotten anything to work. As you can see below (replacing missing values at random between 0-5), you see slight increases across all factors, and the most dramatic increase for Factor 4 - but I think this is because that factor is least well-defined in the Study 1 EFA solution? I think we need to focus on analyzing factor scores from our Study 2 EFA. I’m sorry for this mistake!

Regression models in Study 2 factor space

Here’s a multilevel linear regression on these factor scores, with random intercepts and slopes (for target and factor) by participant. Target is coded as numeric, with only the linear contrast.

If we try to run the model above (our planned analysis), we get an error: “Model is nearly unidentifiable: very large eigenvalue.” The error suggests rescaling variables, which solves the problem. Here I’ve re-scaled by divided age in months by 12, to get age in years. Let’s make sure to talk about this.

Model failed to converge with max|grad| = 0.0388135 (tol = 0.002, component 1)

Linear mixed model fit by REML ['lmerMod']
Formula: score ~ target_num * factor + (target_num + factor | ResponseId)
   Data: efa_all_par_scores %>% mutate(target_num = target_num/12)

REML criterion at convergence: 26763.9

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-7.6050 -0.4547  0.0520  0.5209  4.4723 

Random effects:
 Groups     Name        Variance Std.Dev. Corr                   
 ResponseId (Intercept) 0.37568  0.6129                          
            target_num  0.01556  0.1247   -0.62                  
            factor1     0.18389  0.4288    0.48 -0.35            
            factor2     0.28874  0.5373   -0.53  0.22 -0.42      
            factor3     0.30777  0.5548    0.20  0.27 -0.11 -0.69
 Residual               0.24496  0.4949                          
Number of obs: 15808, groups:  ResponseId, 304

Fixed effects:
                    Estimate Std. Error t value
(Intercept)        -0.369926   0.035549 -10.406
target_num          0.262153   0.007578  34.594
factor1             0.080138   0.026242   3.054
factor2            -0.277623   0.032148  -8.636
factor3             0.263423   0.033108   7.957
target_num:factor1 -0.056791   0.004324 -13.134
target_num:factor2  0.196741   0.004324  45.499
target_num:factor3 -0.186678   0.004324 -43.172
convergence code: 0
Model failed to converge with max|grad| = 0.0388135 (tol = 0.002, component 1)

As we predicted (H2), we see dramatic increases in mental capacity attributions across the age range (main effect of target_num).

And also as we predicted (H1), we see differences across factors in where newborns are perceived to start off: Relative to the grand mean, newborns are perceived to start off with more “negative emotions” (distress, frustration, etc.; main effect of factor1), less/fewer capacities in the domain of “cognition and control” (emotional control, self control, etc.; main effect of factor2), and relatively more “bodily sensations” (pain, fatigue, etc.; main effect of factor3). (We could recode this to look at factor4, or just eyeball it from the plot.) Also as predicted, we see that the perceived changes across age vary dramatically across factors: “negative emotions” are perceived to change relatively less over development, “cognition and control” are perceived to change much more over development, and “bodily sensations” are predicted to chagne relatively less.

This is all very much in line with our preregistered hypotheses :)

Now let’s see what the polynomial effects look like (again, looking at age in years instead of months). As we expected, including all of the polynomial effects as random slopes caused the model not to converge (I think we must be calculating df wrong), so I implemented our preregistered remedy and included only the linear effect as a random slope.

Model failed to converge with max|grad| = 0.254176 (tol = 0.002, component 1)

Linear mixed model fit by REML ['lmerMod']
Formula: score ~ poly(target_num, 3) * factor + (poly(target_num, 1) +  
    factor | ResponseId)
   Data: efa_all_par_scores %>% mutate(target_num = target_num/12)

REML criterion at convergence: 23281.7

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-8.4081 -0.4919  0.0056  0.5500  4.9585 

Random effects:
 Groups     Name                Variance Std.Dev. Corr                   
 ResponseId (Intercept)           0.2762  0.5255                         
            poly(target_num, 1) 623.6907 24.9738  -0.41                  
            factor1               0.1875  0.4330   0.44 -0.35            
            factor2               0.2929  0.5412  -0.54  0.23 -0.43      
            factor3               0.3081  0.5550   0.32  0.25 -0.11 -0.69
 Residual                         0.1928  0.4391                         
Number of obs: 15808, groups:  ResponseId, 304

Fixed effects:
                               Estimate Std. Error t value
(Intercept)                   5.449e-14  3.034e-02   0.000
poly(target_num, 3)1          5.197e+01  1.498e+00  34.691
poly(target_num, 3)2         -2.111e+01  4.391e-01 -48.082
poly(target_num, 3)3          9.471e+00  4.391e-01  21.570
factor1                       4.083e-14  2.556e-02   0.000
factor2                      -3.362e-14  3.162e-02   0.000
factor3                      -3.642e-14  3.240e-02   0.000
poly(target_num, 3)1:factor1 -1.126e+01  7.605e-01 -14.804
poly(target_num, 3)2:factor1  3.958e+00  7.605e-01   5.204
poly(target_num, 3)3:factor1 -2.147e+00  7.605e-01  -2.823
poly(target_num, 3)1:factor2  3.900e+01  7.605e-01  51.285
poly(target_num, 3)2:factor2  2.110e+00  7.605e-01   2.774
poly(target_num, 3)3:factor2 -6.915e+00  7.605e-01  -9.092
poly(target_num, 3)1:factor3 -3.701e+01  7.605e-01 -48.661
poly(target_num, 3)2:factor3  1.306e+01  7.605e-01  17.168
poly(target_num, 3)3:factor3 -5.433e+00  7.605e-01  -7.143

Correlation matrix not shown by default, as p = 16 > 12.
Use print(x, correlation=TRUE)  or
    vcov(x)        if you need it

convergence code: 0
Model failed to converge with max|grad| = 0.254176 (tol = 0.002, component 1)

Lots to sift through here, but in general we see that the effect of target age on mental capacity attributions definitely has linear, quadratic, and cubic components, all three of which seem to vary substantially across factors. Pretty much all of these differences are “significant” (if you consider |t| > 2 to be “significant”) - for interpretation, I would need to look closer at the plot. Let’s pull it up again here, with blue lines approximating the formula y ~ poly(x, 3):

Ignoring unknown aesthetics: y

We can talk through these interpretations together - but I find the difference between Factor 2 (“cognition & control”) and Factor 4 (“positive emotions”) to be especially interesting!

Demographics

Factor `Education` contains implicit NA, consider using `forcats::fct_explicit_na`

Factor `Income` contains implicit NA, consider using `forcats::fct_explicit_na`

Factor `Parent` contains implicit NA, consider using `forcats::fct_explicit_na`

Planning for S3 prereg

Joining, by = "capacity"
Column `capacity` joining character vector and factor, coercing into character vectorJoining, by = "capacity"

unable to evaluate scaled gradientModel failed to converge: degenerate  Hessian with 2 negative eigenvalues

Linear mixed model fit by REML ['lmerMod']
Formula: response ~ target_num * domain + (target_num + domain | ResponseId) +  
    (target_num | capacity)
   Data: d_temp_culled %>% mutate(target_num = target_num/12)

REML criterion at convergence: 276393.6

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-5.5989 -0.4572  0.0289  0.5102  5.4326 

Random effects:
 Groups     Name        Variance Std.Dev. Corr                   
 ResponseId (Intercept) 242.05   15.558                          
            target_num   11.99    3.462   -0.58                  
            domain1     115.42   10.743   -0.48  0.49            
            domain2     198.33   14.083   -0.28  0.15 -0.43      
            domain3     202.84   14.242    0.58 -0.32 -0.21 -0.58
 capacity   (Intercept) 454.81   21.326                          
            target_num  315.86   17.773   -0.17                  
 Residual               320.25   17.896                          
Number of obs: 31616, groups:  ResponseId, 304; capacity, 8

Fixed effects:
                   Estimate Std. Error t value
(Intercept)          57.758      7.594   7.606
target_num            7.579      6.287   1.206
domain1              33.744     13.076   2.581
domain2             -45.589     13.087  -3.484
domain3               7.296     13.087   0.557
target_num:domain1   -6.308     10.884  -0.580
target_num:domain2    6.709     10.884   0.616
target_num:domain3   -1.445     10.884  -0.133
convergence code: 0
unable to evaluate scaled gradient
Model failed to converge: degenerate  Hessian with 2 negative eigenvalues

Model failed to converge with max|grad| = 0.0296471 (tol = 0.002, component 1)

Linear mixed model fit by REML ['lmerMod']
Formula: response ~ target_num * domain + (target_num + domain | ResponseId) +  
    (1 | capacity)
   Data: d_temp_culled %>% mutate(target_num = target_num/12)

REML criterion at convergence: 276844.7

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-5.5281 -0.4507  0.0443  0.5053  5.4042 

Random effects:
 Groups     Name        Variance Std.Dev. Corr                   
 ResponseId (Intercept) 241.25   15.532                          
            target_num   11.99    3.463   -0.58                  
            domain1     115.76   10.759   -0.48  0.49            
            domain2     199.04   14.108   -0.28  0.15 -0.44      
            domain3     202.81   14.241    0.58 -0.32 -0.21 -0.57
 capacity   (Intercept)  50.80    7.127                          
 Residual               325.55   18.043                          
Number of obs: 31616, groups:  ResponseId, 304; capacity, 8

Fixed effects:
                   Estimate Std. Error t value
(Intercept)         57.7579     2.6762  21.582
target_num           7.5794     0.2088  36.302
domain1             33.7440     4.4143   7.644
domain2            -45.5894     4.4452 -10.256
domain3              7.2957     4.4466   1.641
target_num:domain1  -6.3077     0.1115 -56.590
target_num:domain2   6.7086     0.1115  60.186
target_num:domain3  -1.4450     0.1115 -12.963
convergence code: 0
Model failed to converge with max|grad| = 0.0296471 (tol = 0.002, component 1)

Linear mixed model fit by REML ['lmerMod']
Formula: score ~ target_num * domain + (target_num | ResponseId)
   Data: d_temp_culled %>% mutate(target_num = target_num/12) %>% group_by(domain,  
    target_num, ResponseId) %>% summarise(score = mean(response,  
    na.rm = T)) %>% ungroup() %>% distinct()

REML criterion at convergence: 138809.4

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.7636 -0.5695  0.0200  0.6602  4.9951 

Random effects:
 Groups     Name        Variance Std.Dev. Corr 
 ResponseId (Intercept) 234.89   15.326        
            target_num   10.57    3.251   -0.58
 Residual               347.42   18.639        
Number of obs: 15808, groups:  ResponseId, 304

Fixed effects:
                   Estimate Std. Error  t value
(Intercept)         57.7579     0.9013   64.086
target_num           7.5794     0.2088   36.298
domain1             33.7440     0.3446   97.928
domain2            -45.5894     0.3446 -132.304
domain3              7.2957     0.3446   21.173
target_num:domain1  -6.3077     0.1628  -38.735
target_num:domain2   6.7086     0.1628   41.197
target_num:domain3  -1.4450     0.1628   -8.873

Export data

(remember to move prior to pushing to github!)

additional cleaning of raw data